Multi-view images (and videos) typically refer to a set of overlapped images capturing a scene from different view positions. One or more cameras can be employed to take multi-view images. One common approach in compressing multi-view images is to encode the images (and videos) from different viewpoints independently. However, this approach does not exploit the correlation between different views and often results in an enormous amount of redundant data to be transmitted, or stored in a storage device. An alternative approach is to take advantage of the inter-view correlation when encoding. With this encoding approach, an encoder reads the captured images from all different viewpoints and performs inter-view predictive coding on the captured images. This coding scheme can achieve high coding efficiency at the expense of a high computational complexity encoder.
Wyner-Ziv coding, or “distributed video coding”, has recently been extended to stereo and multi-view imaging to address the shortcomings of conventional coding approaches. In a distributed video coding (DVC) scheme, the complexity is shifted from the encoder to the decoder. Typically, the set of input images captured at different viewpoints at the same time instant is usually split into two subsets. The first subset of images has at least one image and is compressed using a conventional coding scheme, such as JPEG, JPEG2000, and H.264 (Intra), which can be decoded conventionally at the decoder. The second subset of images is encoded using channel coding methods and requires a joint decoding process to reconstruct its image content. The joint decoding process is performed in two main steps. Firstly, the second subset of images is predicted from the conventionally decoded images of the first subset based on the inter-view correlation. Secondly, the error correction information associated to the second subset of images is applied to the predicted images to improve the visual quality of the predicted images.
One major challenge for multi-view imaging systems that employ distributed to coding techniques is the quality of the inter-view prediction. A multi-view imaging system often produces low-quality inter-view prediction when:
(i) the imaging devices (i.e., cameras) at different viewpoints are not carefully calibrated;
(ii) the images captured at different viewpoints have mixed resolutions;
(iii) the image planes of all imaging devices at different viewpoints are not arranged so that they are coplanar (e.g., as a camera array); and
(iv) the optical axes of the imaging devices at different viewpoints are not perfectly parallel to each other.